Optimal Importance Sampling in Markov Process Simulation
نویسندگان
چکیده
In this paper we present an adaptive algorithm to estimate the transient blocking probability of a communication system, described by a Markov process, during a finite time interval starting from a given state. The method uses importance sampling for variance reduction and adjusts the parameters of the twisted distribution based on earlier samples. The method can be effectively applied to a decision making problem where future revenues are estimated with extensive simulations, in order to find an improved policy by so called first policy iteration.
منابع مشابه
Dynamic Importance Sampling for Uniformly Recurrent Markov Chains
Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested by a large deviation lower bound analysis. Recent work, however, has suggested that such schemes do not work well in many situations. In this paper we consid...
متن کاملEfficient Simulation of Fluid Queues with Many Sources
In this paper we study the rare event of overflow in a Markov fluid queue with finite buffer and many input sources. The probability of this rare event will be estimated by simulations. We present a highly efficient importance sampling procedure to speed up the simulations. The implemented change of meausure is suggested after a large deviations analysis of the overflow probability. This analys...
متن کاملAn Optimal Importance Sampling Method for a Transient Markov System
In this paper an optimal importance sampling (IS) method is derived for a transient markov system. Several propositions are presented. It is showned that the optimal IS method is unique, and it must converge to the standard Monte Carlo (MC) simulation method when the sample path length approaches infinity. Therefore, it is not the size of the state space of the Markov system, but the sample pat...
متن کاملOn the optimal Markov chain of IS simulation
We investigate the importance sampling (IS) simulation for the sample average of an output sequence from an irreducible Markov chain. The optimal Markov chain used in simulation is known to be a twisted Markov chain, however, the proofs in [2], [3] are very complicated and do not give us a good perspective. We give a simple and natural proof for the optimality of the simulation Markov chain in ...
متن کاملExamples comparing Importance Sampling and the Metropolis algorithm
Importance sampling, particularly sequential and adaptive importance sampling, have emerged as competitive simulation techniques to Markov–chain Monte–Carlo techniques. We compare importance sampling and the Metropolis algorithm as two ways of changing the output of a Markov chain to get a different stationary distribution.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2001